CONTINUAL LEARNING

Continual Prompt Tuning for Dialog State Tracking (2022.acl-long)

Challenge:	Existing methods to train LLMs on previous training data are not feasible in real-world applications because of catastrophic forgetting.
Approach:	They propose a framework that uses the LLM to generate synthetic instances for rehearsal and refine the instance outputs based on the synthetic inputs.
Outcome:	The proposed framework achieves superior or comparable performance compared to conventional rehearsal-based approaches while being more data-efficient.

Copied to clipboard

Qi Zhu, Bing Li, Fei Mi, Xiaoyan Zhu, Minlie Huang

Responsible & Trustworthy NLP

Challenge:	Existing methods to train a model on a sequence of tasks are not efficient enough to mitigate catastrophic forgetting.
Approach:	They propose a parameter-efficient framework that prevents forgetting and enables knowledge transfer between tasks by learning and freezing a pre-trained model.
Outcome:	The proposed framework avoids forgetting and enables knowledge transfer between tasks.

Prompt Augmented Generative Replay via Supervised Contrastive Learning for Lifelong Intent Detection (2022.findings-naacl)

Copied to clipboard

Vaibhav Varshney, Mayur Patidar, Rajat Kumar, Lovekesh Vig, Gautam Shroff

ELLA: Efficient Lifelong Learning for Adapters in Large Language Models (2026.eacl-long)

Challenge:	Existing methods to identify all possible user intents at design time are expensive and require storage of past data.
Approach:	They propose to continually train an intent detector on new intents while maintaining performance on prior intents.
Outcome:	The proposed method outperforms exemplar replay-based approaches on lifelong intent detection tasks and achieves state-of-the-art on four public datasets.

Copied to clipboard

Shristi Das Biswas, Yue Zhang, Anwesan Pal, Radhika Bhargava, Kaushik Roy

Towards Continual Learning for Multilingual Machine Translation via Vocabulary Substitution (2021.naacl-main)

Challenge:	Existing approaches to training Large Language Models (LLMs) suffer from catastrophic forgetting when adapted sequentially to new tasks in a continual learning (CL) setting. Existing methods are impractical and could potentially violate privacy.
Approach:	They propose a training framework built on the principle of selective subspace de-correlation that characterizes the structure of past updates and penalizes alignments along their high-energy, task-specific directions.
Outcome:	The proposed training framework achieves state-of-the-art CL performance on three popular benchmarks spanning both classification and generative tasks with relative accuracy gains of up to 9.6% and a 35 smaller memory footprint.

Copied to clipboard

Xavier Garcia, Noah Constant, Ankur Parikh, Orhan Firat

Multilinguality

Challenge:	Existing approaches to multilingual machine translation rely on training models on monolingual data for all languages in a multitask setup.
Approach:	They propose a vocabulary adaptation scheme to extend the language capacity of multilingual machine translation models by combining monolingual data with a dictionary.
Outcome:	The proposed model improves on existing models by preserving the original model and allowing for competitive performance even with only monolingual data.

Class-Incremental Learning based on Label Generation (2023.acl-short)

Copied to clipboard

Yijia Shao, Yiduo Guo, Dongyan Zhao, Bing Liu

Rethinking Long Context Generation from the Continual Learning Perspective (2025.coling-main)

Challenge:	Existing studies on pre-trained language models focus on task-incremental learning (TIL) but they perform poorly in a more challenging setting of class-incremental learning.
Approach:	They propose a method which solves CIL based on label generation by using sparse vocabulary and creates pseudo-replay samples by using label semantics.
Outcome:	The proposed method outperforms baseline models by a large margin in the class-incremental learning setting.

Copied to clipboard

Zeyuan Yang, Fangzhou Xiong, Peng Li, Yang Liu

Text Generation

Challenge:	Large Language Models (LLMs) struggle with processing long contexts due to the limited context window.
Approach:	They propose to combine a limited context window with a continual learning perspective to improve LLMs' efficiency in processing long contexts.
Outcome:	The proposed models improve the performance of Large Language Models (LLMs) by integrating learning strategies with existing approaches.

Overcoming Catastrophic Forgetting beyond Continual Learning: Balanced Training for Neural Machine Translation (2022.acl-long)

Copied to clipboard

Chenze Shao, Yang Feng

Machine Translation

Challenge:	Neural networks tend to gradually forget the previously learned knowledge when learning multiple tasks sequentially from dynamic data distributions.
Approach:	They propose a method that iteratively provides complementary knowledge to student models by dynamically updating teacher models trained on specific data orders.
Outcome:	The proposed method improves on multiple machine translation tasks and improves performance over baseline systems.

MTIVE: Multi-Task Image Verification Engine Using Vision-Language Models for E-commerce (2026.acl-industry)

Copied to clipboard

Yu-Tong Cao, Vishnu Prabhakaran, Arunita Das, Purav Aggarwal, Anoop Saladi

Visually Grounded Continual Learning of Compositional Phrases (2020.emnlp-main)

Challenge:	Vision-language models struggle with noisy real-world images and multi-task requirements.
Approach:	They propose a curriculum learning framework that adapts vision-language models through three stages . MTIVE uses frozen base weights with stacked LoRA adapters for shared domain knowledge .
Outcome:	MTIVE outperforms open-source and proprietary baselines in standard and continual learning settings.

Copied to clipboard

Xisen Jin, Junyi Du, Arka Sadhu, Ram Nevatia, Xiang Ren

Multimodality

Challenge:	Modern NLP systems rely on offline training and are inefficient for new tasks.
Approach:	They propose a visually grounded ContinuaL learning task which simulates the continual acquisition of compositional phrases from streaming visual scenes.
Outcome:	The proposed system improves on existing systems, but it's infeasible to store all possible compositions.

ChronosLex: Time-aware Incremental Training for Temporal Generalization of Legal Classification Tasks (2024.acl-long)

Copied to clipboard

Santosh T.y.s.s, Tuan-Quang Vuong, Matthias Grabmair

Information Retrieval

Challenge:	Existing models overlook the temporal dimension in their training process, leading to suboptimal performance over time.
Approach:	They propose a training paradigm that trains models on chronological splits, preserving the temporal order of the data.
Outcome:	The proposed model fails to fit to recent data, despite continual learning and temporal invariant methods.

Non-compositional Expression Generation and its Continual Learning (2024.findings-acl)

Copied to clipboard

Jianing Zhou, Suma Bhat

Learn and Review: Enhancing Continual Named Entity Recognition via Reviewing Synthetic Samples (2022.findings-acl)

Challenge:	Recent work shows that pre-trained language models are limited in their ability to generate non-compositional expressions.
Approach:	They propose a mask-infilling task to examine non-compositional expressions in English . they compare large pre-trained language models and continual learning methods .
Outcome:	The proposed task aims to investigate the ability of pre-trained language models to generate non-compositional expressions in English and their continual learning.

Copied to clipboard

Yu Xia, Quan Wang, Yajuan Lyu, Yong Zhu, Wenhao Wu, Sujian Li, Dai Dai

Text Classification Under Class Distribution Shift: A Survey (2026.eacl-long)

Challenge:	Existing methods for named entity recognition classify mentions into fixed set of predefined entity types but in many real-world scenarios, new entity types are incrementally involved.
Approach:	They propose a two-stage framework Learn-and-Review for continual named entity recognition to alleviate inter-type confusion.
Outcome:	The proposed framework outperforms the state-of-the-art method on CoNLL-03 and OntoNotes-5.0.

Copied to clipboard

Adriana Valentina Costache, Silviu-Florin Gheorghe, Eduard Poesina, Paul Irofti, Radu Tudor Ionescu

SharpSeq: Empowering Continual Event Detection through Sharpness-Aware Sequential-task Learning (2024.naacl-long)

Challenge:	ML models assume that training and test data are sampled from the same distribution, but in daily practice, this assumption is often broken.
Approach:	They survey articles studying open-set text classification to understand the distribution shifts and mitigation approaches for each problem setup.
Outcome:	The proposed methods can solve problems caused by the shifting class distribution in open-set text classification and related tasks.

Copied to clipboard

Thanh-Thien Le, Viet Dao, Linh Nguyen, Thi-Nhung Nguyen, Linh Ngo, Thien Nguyen

Continual Learning for Text Classification with Information Disentanglement Based Regularization (2021.naacl-main)

Challenge:	Existing methods for continual event detection suffer from catastrophic forgetting . a novel continual learning paradigm leveraging sharpness-aware minimization is needed .
Approach:	They propose a continual learning paradigm that leverages sharpness-aware minimization and a generative model to balance training data distribution.
Outcome:	The proposed approach outperforms existing methods on real-world datasets.

Copied to clipboard

Yufan Huang, Yanzhe Zhang, Jiaao Chen, Xuezhi Wang, Diyi Yang

On Continual Model Refinement in Out-of-Distribution Data Streams (2022.acl-long)

Challenge:	Existing continual learning methods focus on preserving knowledge from previous tasks . Continual learning is a useful tool for learning over time, but it is not always possible to generalize to new tasks.
Approach:	They propose a disentanglement-based regularization method for continual learning on text classification that disentangles text hidden spaces into generic representations and regularizes them differently to constrain knowledge required to generalize.
Outcome:	The proposed method disentangles text hidden spaces into representations that are generic to all tasks and representations specific to each individual task.

Copied to clipboard

Bill Yuchen Lin, Sida Wang, Xi Lin, Robin Jia, Lin Xiao, Xiang Ren, Scott Yih

Challenge:	Existing continual learning (CL) problems cannot cover real-world scenarios such as out-of-distribution errors.
Approach:	They propose a continual model refinement problem formulation to solve this problem . they extend several existing continual learning approaches to the CMR problem based on a general sampling algorithm .
Outcome:	The proposed model refinement solution improves on existing models and their performance metrics.

Continual Dialogue State Tracking via Example-Guided Question Answering (2023.emnlp-main)

Copied to clipboard

Hyundong Cho, Andrea Madotto, Zhaojiang Lin, Khyathi Chandu, Satwik Kottur, Jing Xu, Jonathan May, Chinnadhurai Sankar

Natural Language Interfaces

Challenge:	Dialogue systems are frequently updated to accommodate new services, but naively updating them by continually training with data for new services causes catastrophic forgetting.
Approach:	They propose to reformulate dialogue state tracking (DST) as a bundle of example-guided question answering tasks to minimize the task shift between services.
Outcome:	The proposed model achieves state-of-the-art performance on DST continual learning metrics without relying on any complex regularization or parameter expansion methods.

Continued Pretraining and Interpretability-Based Evaluation for Low-Resource Languages: A Galician Case Study (2025.findings-acl)

Copied to clipboard

Pablo Rodríguez, Silvia Paniagua Suárez, Pablo Gamallo, Susana Sotelo Docio

Responsible & Trustworthy NLP

Challenge:	Recent advances in large language models have led to remarkable improvements in language understanding and text generation.
Approach:	They propose a framework to evaluate large language models for underrepresented languages . they examine CPT strategies for languages with limited representation in multilingual models .
Outcome:	The proposed evaluation framework is based on the case of Galician language . it assesses trade-offs between linguistic enrichment and task-solving capabilities .

SLoRA: Balancing Plasticity and Forgetting in Large Language Models for Continual Learning (2026.acl-long)

Copied to clipboard

Lina Yang, Yusheng Liao, Yanfeng Wang, Yu Wang

Non-compositional Expression Generation Based on Curriculum Learning and Continual Learning (2023.findings-emnlp)

Challenge:	Large language models (LLMs) have achieved remarkable success across diverse tasks through large-scale pretraining.
Approach:	They propose a framework that filters noisy components from LoRA updates via subspace similarity with the base model.
Outcome:	The proposed framework improves accuracy by 12%, reduces forgetting by 29%, and filters out over 30% of LoRA parameters identified as noisy.

Copied to clipboard

Jianing Zhou, Ziheng Zeng, Hongyu Gong, Suma Bhat

Is Parameter Collision Hindering Continual Learning in LLMs? (2025.coling-main)

Challenge:	Non-compositional expressions are a classic ‘pain in the neck’ for NLP systems because of their non-composibility and limited data resources.
Approach:	They propose a dynamic curriculum learning framework which learns training examples from easy ones to harder ones but suffers from the forgetting problem.
Outcome:	The proposed framework improves on idiomatic expression generation and metaphor generation.

Copied to clipboard

Shuo Yang, Kun-Peng Ning, Yu-Yang Liu, Jia-Yu Yao, Yong-Hong Tian, Yi-Bing Song, Li Yuan

Total Recall: a Customized Continual Learning Method for Neural Semantic Parsers (2021.emnlp-main)

Challenge:	Existing methods to learn multiple tasks in parallel often lead to catastrophic forgetting, resulting in overwriting knowledge.
Approach:	They propose a non-collision low-rank Adaptation approach that leverages low collision rates to enhance continual learning (CL) in large language models.
Outcome:	The proposed approach achieves better task orthogonality and higher task orthognality than existing SOTA methods.

Copied to clipboard

Zhuang Li, Lizhen Qu, Gholamreza Haffari

Continual Neural Topic Model (2026.eacl-long)

Challenge:	Existing methods for continual learning for semantic parsing fail to account for special properties of structured outputs . retraining from scratch is not feasible due to the fast growing number of tasks .
Approach:	They propose a continual learning method that uses sequential learning to learn tasks without accessing full training data from previous tasks.
Outcome:	The proposed method achieves a 3-6 times speedup compared to re-training from scratch.

Copied to clipboard

Charu Karakkaparambil James, Waleed Mustafa, Marcio Monteiro, Marius Kloft, Sophie Fellenz

Psycholinguistics Meets Continual Learning: Measuring Catastrophic Forgetting in Visual Question Answering (P19-1)

Challenge:	Continual Neural Topic Models (CoNTMs) learn topic models at subsequent time steps without forgetting what was previously learned.
Approach:	They propose a Continual Neural Topic Model which continuously learns topic models at subsequent time steps without forgetting what was previously learned.
Outcome:	The proposed model outperforms the dynamic topic model in topic quality and predictive perplexity while being able to capture topic changes online.

Copied to clipboard

Claudio Greco, Barbara Plank, Raquel Fernández, Raffaella Bernardi

Visual Data in NLP

Challenge:	Existing methods to overcome catastrophic forgetting in visual question answering models are inadequate, but have received little attention within natural language processing.
Approach:	They devise a set of linguistically-informed visual question answering tasks motivated by psycholinguistics and investigate impact of task difficulty on continual learning.
Outcome:	The proposed models differ in the types of questions they ask and show that task difficulty and order matter.

Unlocking Continual Learning Abilities in Language Models (2024.findings-emnlp)

Copied to clipboard

Wenyu Du, Shuang Cheng, Tongxu Luo, Zihan Qiu, Zeyu Huang, Ka Chun Cheung, Reynold Cheng, Jie Fu

SEE: Continual Fine-tuning with Sequential Ensemble of Experts (2025.findings-acl)

Challenge:	Existing approaches to learning models (LMs) incorporate old task data or task-wise inductive bias into LMs, but old data and accurate task information are often unavailable or costly to collect.
Approach:	They propose a rehearsal-free method that updates model parameters with large magnitudes . they found that the L1-normalized magnitude distribution is different when different task data is used .
Outcome:	The proposed method improves accuracy and performance on four CL benchmarks.

Copied to clipboard

Zhilin Wang, Yafu Li, Xiaoye Qu, Yu Cheng

Continual Learning Using Only Large Language Model Prompting (2025.coling-main)

Challenge:	Continual fine-tuning of large language models suffers from catastrophic forgetting . some approaches use routers to assign tasks to experts, but continual learning often requires retraining .
Approach:	They propose a framework that integrates routing and response mechanisms within each expert . it eliminates the need for an additional router and allows each expert to decide whether a query should be handled .
Outcome:	The proposed framework outperforms previous approaches in continual fine-tuning . it can handle learning tasks and out-of-distribution instances, paving the way for distributed model ensembling.

Copied to clipboard

Jiabao Qiu, Zixuan Ke, Bing Liu

SKD-NER: Continual Named Entity Recognition via Span-based Knowledge Distillation with Reinforcement Learning (2023.emnlp-main)

Challenge:	Existing continuous learning paradigms fine-tune language model parameters or use adapters or variants to adapt the LM.
Approach:	They propose a new continual learning paradigm wherein a large language model is regarded as a black box.
Outcome:	The proposed method outperforms baselines by a large margin in learning tasks incrementally.

Copied to clipboard

Yi Chen, Liang He

Lifelong Sequence Generation with Dynamic Module Expansion and Adaptation (2023.emnlp-main)

Challenge:	Continual learning for named entity recognition (CL-NER) aims to enable models to continuously learn new entity types while retaining the ability to recognize previously learned ones.
Approach:	They propose a model that leverages knowledge distillation to retain memory and employs reinforcement learning strategies to optimize the soft labeling and distillation losses generated by the teacher model to effectively prevent catastrophic forgetting.
Outcome:	The proposed model outperforms state-of-the-art models on two benchmark datasets showing that it significantly improves the performance of the CL-NER task.

Copied to clipboard

Chengwei Qin, Chen Chen, Shafiq Joty

Text Generation

Challenge:	Existing methods focus on maintaining old knowledge while paying little attention to knowledge transfer across tasks.
Approach:	They propose to train a model on a sequence of generation tasks to learn new generation patterns while avoiding the forgetting of previous knowledge.
Outcome:	The proposed model outperforms existing methods in different settings.

Depth Aware Hierarchical Replay Continual Learning for Knowledge Based Question Answering (2024.lrec-main)

Copied to clipboard

Zhixiong Cao, Hai-Tao Zheng, Yangning Li, Jin Xu, Rongsheng Li, Hong-Gee Kim

Information Retrieval

Challenge:	Continual learning models adapt well to the latest data but lose ability to remember past data due to changes in the data source.
Approach:	They propose a hierarchical replay framework that allows models to keep a small memory of previous learned data that uses replay.
Outcome:	The proposed model outperforms previous continual learning methods in mitigating catastrophic forgetting.

TemporalWiki: A Lifelong Benchmark for Training and Evaluating Ever-Evolving Language Models (2022.emnlp-main)

Copied to clipboard

Joel Jang, Seonghyeon Ye, Changho Lee, Sohee Yang, Joongbo Shin, Janghoon Han, Gyeonghun Kim, Minjoon Seo

Continual Dialogue State Tracking via Reason-of-Select Distillation (2024.findings-acl)

Challenge:	Language Models (LMs) become outdated as the world changes, a phenomenon called temporal misalignment.
Approach:	They propose a lifelong benchmark that utilizes the difference between consecutive snapshots of English Wikipedia and English Wikidata for training and evaluation.
Outcome:	The proposed benchmark can be trained on the difference between consecutive snapshots of English Wikipedia and English Wikidata for training and evaluation.

Copied to clipboard

Yujie Feng, Bo Liu, Xiaoyu Dong, Zexin Lu, Li-Ming Zhan, Xiao-Ming Wu, Albert Lam

Dialogue Systems & Conversational Agents

Challenge:	Existing research on dialogue systems has focused on domain-specific offline systems lacking adaptation abilities.
Approach:	They propose a Reason-of-Select distillation method that enhances smaller models with a novel "meta-reasoning" capability.
Outcome:	Experiments show that the proposed method significantly improves the performance and generalization capabilities of existing models.

In-context Continual Learning Assisted by an External Continual Learner (2025.coling-main)

Copied to clipboard

Saleh Momeni, Sahisnu Mazumder, Zixuan Ke, Bing Liu

MixLLM: Dynamic Routing in Mixed Large Language Models (2025.naacl-long)

Challenge:	Existing continual learning methods suffer from catastrophic forgetting (CF) . Existing methods rely on fine-tuning or adapting large language models (LLMs)
Approach:	They propose an approach that integrates an external continual learner (ECL) with ICL to enable scalable CL without catastrophic forgetting (CF).
Outcome:	The proposed approach outperforms existing baselines while maintaining high performance.

Copied to clipboard

Xinyuan Wang, Yanchi Liu, Wei Cheng, Xujiang Zhao, Zhengzhang Chen, Wenchao Yu, Yanjie Fu, Haifeng Chen

Continual Learning in Multilingual Sign Language Translation (2025.naacl-long)

Challenge:	Large Language Models (LLMs) exhibit potential artificial generic intelligence, however, their usage is costly with high response latency.
Approach:	They develop a dynamic contextual-bandit-based routing system for query-LLM assignment that leverages query tags to enhance query embeddings.
Outcome:	The proposed model maximizes response quality and minimizes cost and latency.

Copied to clipboard

Shakib Yazdani, Josef Van Genabith, Cristina España-Bonet

Multilinguality

Challenge:	Despite the low translation quality of sign language, many machine learning approaches are still in its infancy.
Approach:	They propose to use continual learning for mul- tilingual SLT to improve translation quality.
Outcome:	The proposed methods outperform baseline and fine-tuning approaches in sign language translation.

Dynamic-prototype Contrastive Fine-tuning for Continual Few-shot Relation Extraction with Unseen Relation Detection (2025.coling-main)

Copied to clipboard

Si Miao Zhao, Zhen Tan, Ning Pang, Wei Dong Xiao, Xiang Zhao

Continual Learning in Task-Oriented Dialogue Systems (2021.emnlp-main)

Challenge:	Existing approaches to learn relations from labeled data overlook task interference in continual learning and memory requirements for different relations.
Approach:	They propose a framework to learn new relations from limited labeled data while preserving knowledge about previously learned relations.
Outcome:	The proposed framework is more practical and comprehensive for real-world scenarios.

Copied to clipboard

Andrea Madotto, Zhaojiang Lin, Zhenpeng Zhou, Seungwhan Moon, Paul Crook, Bing Liu, Zhou Yu, Eunjoon Cho, Pascale Fung, Zhiguang Wang

Natural Language Interfaces

Challenge:	Existing continuous learning systems are not designed to add new domains and functionalities through time without incurring the high cost of retraining the whole system.
Approach:	They propose a first-ever continual learning benchmark for task-oriented dialogue systems . they propose 'architecture' method based on residual adapters to implement continual training .
Outcome:	The proposed architectural method performs better than multitask learning while being 20X faster in learning new domains.

SAPT: A Shared Attention Framework for Parameter-Efficient Continual Learning of Large Language Models (2024.acl-long)

Copied to clipboard

Weixiang Zhao, Shilong Wang, Yulin Hu, Yanyan Zhao, Bing Qin, Xuanyu Zhang, Qing Yang, Dongliang Xu, Wanxiang Che

HFT: Half Fine-Tuning for Large Language Models (2025.acl-long)

Challenge:	Existing methods to address catastrophic forgetting and knowledge transfer in large language models (LLMs) ignore potential of aligning the two modules to effectively address catastrophic forgetting and knowledge transfers simultaneously.
Approach:	They propose a Shared Attentive Learning & Selection module to align the PET learning and selection modules to address catastrophic forgetting and knowledge transfer simultaneously.
Outcome:	Experiments on two CL benchmarks show that the proposed framework is superior when scaled to different model sizes, different model architectures and unseen tasks.

Copied to clipboard

Tingfeng Hui, Zhenyu Zhang, Shuohuan Wang, Weiran Xu, Yu Sun, Hua Wu

Prototype Conditioned Generative Replay for Continual Learning in NLP (2025.naacl-long)

Challenge:	Large language models (LLMs) with one or more fine-tuning phases can unlock various capabilities, but can be catastrophic forgetting during sequential training.
Approach:	They propose a method to regularly reset partial parameters to mitigate forgetting issues by using half fine-tuning instead of full fine-uning.
Outcome:	The proposed approach reduces the risk of catastrophic forgetting during training and the parametric knowledge lost during training may be overwhelmed by incoming training data.

Copied to clipboard

Xi Chen, Min Zeng

Challenge:	Generative replay methods that rely on a single task-specific token or prompt often fail to generate pseudo-samples that accurately reflect the true data distribution.
Approach:	They propose a Prototype Conditioned Generative Replay method which incorporates task-level statistics into a prototyping process.
Outcome:	The proposed method outperforms state-of-the-art (SOTA) methods on two different scenarios.

Merge then Realign: Simple and Effective Modality-Incremental Continual Learning for Multimodal LLMs (2025.emnlp-main)

Copied to clipboard

Dingkun Zhang, Shuhan Qi, Xinyu Xiao, Kehai Chen, Xuan Wang

Multimodality

Challenge:	Recent advances in Multimodal Large Language Models (MLLMs) have enhanced their versatility as they integrate a growing number of modalities.
Approach:	They propose a simple MCL paradigm that addresses forgetting and misalignment . they propose 'MErge then ReAlign' to extend existing models to more modalities .
Outcome:	The proposed paradigm is easy to deploy and highly reusable in the MLLM community.

TL-CL: Task And Language Incremental Continual Learning (2024.emnlp-main)

Copied to clipboard

Shrey Satapara, P. K. Srijith

Green & Sustainable NLP

Challenge:	a multilingual model is periodically updated to accommodate new tasks in previously learned languages or new languages for established tasks.
Approach:	They propose an adapter-based parameter-efficient fine-tuning strategy for continual learning in multilingual models.
Outcome:	The proposed approach outperforms other parameter-efficient approaches without access to historical data for replay.

Improving Continual Few-shot Relation Extraction through Relational Knowledge Distillation and Prototype Augmentation (2024.lrec-main)

Copied to clipboard

Zhiheng Zhang, Daojian Zeng, Xue Bai

Responsible & Trustworthy NLP

Challenge:	Existing approaches to Continual Relation Extraction (CRE) are limited in handling the rapid emergence of new relations in real-world scenarios.
Approach:	They propose a framework that integrates prototype-based data augmentation and relational knowledge distillation to solve the problem of Continual Few-shot Relation Extraction (CFRE).
Outcome:	The proposed framework outperforms the state-of-the-art methods on the FewRel and TACRED datasets.

RECALL: REpresentation-aligned Catastrophic-forgetting ALLeviation via Hierarchical Model Merging (2025.emnlp-main)

Copied to clipboard

Bowen Wang, Haiyuan Wan, Liwen Shi, Chen Yang, Peng He, Yue Ma, Haochen Han, Wenhao Li, Tiao Tan, Yongjian Li, Fangming Liu, Gong Yifan, Sheng Zhang

Soft Orthogonal Low-Rank Adaptation for Knowledge Sharing in Large Language Model Continual Learning (2026.acl-long)

Challenge:	Existing models that require task labels or performance trade-offs are susceptible to catastrophic forgetting.
Approach:	They propose a representation-aware model merging framework for continual learning without access to historical data.
Outcome:	The proposed framework outperforms baselines in knowledge retention and generalization across five NLP tasks and multiple continual learning scenarios.

Copied to clipboard

Yitong Wang, Xue Han, WenChun Gao, Qian Hu, Jiahui Wang, Ziqing Wang, Lijun Mei, Junlan Feng

Explicit Memory Learning with Expectation Maximization (2024.emnlp-main)

Challenge:	Existing methods for continual learning (CL) are designed to mitigate catastrophic forgetting while neglecting knowledge sharing across tasks.
Approach:	They propose a framework that facilitates knowledge transfer while mitigating catastrophic forgetting by assigning task-specific parameter subspaces to new tasks . they then leverage attribution scores to evaluate task similarity and employ soft orthogonality between task- specific subspace .
Outcome:	The proposed framework facilitates knowledge transfer while mitigating catastrophic forgetting.

Copied to clipboard

Zhangyue Yin, Qiushi Sun, Qipeng Guo, Zhiyuan Zeng, Qinyuan Cheng, Xipeng Qiu, Xuanjing Huang

Challenge:	Large Language Models lack reliable learning mechanisms for updating information across interactions.
Approach:	They propose a framework that enhances explicit memory updates via the Expectation-Maximization algorithm.
Outcome:	The proposed framework outperforms existing methods without memory or with static external memory on streaming inference tasks.

SCALE: Synergized Collaboration of Asymmetric Language Translation Engines (2024.findings-acl)

Copied to clipboard

Xin Cheng, Xun Wang, Tao Ge, Si-Qing Chen, Furu Wei, Dongyan Zhao, Rui Yan

Multilinguality

Challenge:	In this paper, we introduce SCALE, a collaborative framework that connects a compact Specialized Translation Model (STM) and a general-purpose Large Language Model (LLM) as one unified translation engine.
Approach:	They propose a collaborative framework that connects a Specialized Translation Model (STM) and a general-purpose Large Language Model (LLM) as one unified translation engine.
Outcome:	The proposed framework outperforms both LLMs and supervised models in high-resource or challenging low-resourced settings.

Unveiling and Addressing Pseudo Forgetting in Large Language Models (2025.findings-acl)

Copied to clipboard

Huashan Sun, Yizhe Yang, Yinghao Li, Jiawei Li, Yang Gao

Empowering Math Problem Generation and Reasoning for Large Language Model via Synthetic Data based Continual Learning Framework (2025.emnlp-main)

Challenge:	Existing efforts to mitigate catastrophic forgetting in continual learning have not been studied.
Approach:	They propose a rationale-guided replay framework that allows models to leverage their capabilities and provide partial external correct rationales to the original instructions.
Outcome:	The proposed framework mitigates pseudo forgetting while maintaining model plasticity.

Copied to clipboard

Qian Wan, Wangzi Shi, Jintian Feng, Shengyingjie Liu, Luona Wei, Zhicheng Dai, Jianwen Sun

Reasoning

Challenge:	Existing learning frameworks for large language models (LLMs) for math problem generation are limited and lack quality data.
Approach:	They propose a synthetic data based continual learning framework to improve LLMs ability for MPG and math reasoning.
Outcome:	The proposed framework improves performance on large language models and math reasoning using supervised fine-tuning, data synthesis and direct preference optimization.

Harmonizing the Past, Present, and Future: A Null-Space Constrained Region-Specific Method for Continual Learning in LLMs (2026.acl-long)

Copied to clipboard

Jinhui Chen, Shizhu He, Xingchang Yang, Huanxuan Liao, Yequan Wang, Xiangwen Liao, Wenhao Teng, Kang Liu, Jun Zhao

Towards Autonomous Tool Utilization in Language Models: A Unified, Efficient and Scalable Framework (2024.lrec-main)

Challenge:	Existing continual learning paradigms prioritize instant performance through dense updates, leading to catastrophic forgetting and rapid exhaustion of model capacity.
Approach:	They propose a method that preserves previously acquired knowledge and acquires new task-specific skills while preserving sufficient parameter capacity for subsequent adaptation.
Outcome:	The proposed method is based on the brain's functional partitioning and can be used to map tasks between specialized and generalist neurons.

Copied to clipboard

Zhi Li, Yicheng Li, Hequan Ye, Yin Zhang

CL2GEC: A Multi-Discipline Benchmark for Continual Learning in Chinese Literature Grammatical Error Correction (2026.acl-long)

Challenge:	Recent advances in tool learning for large language models have led to a new trend to allow LLMs to leverage external tools.
Approach:	They propose a framework for fine-tuning language models that categorizes queries into three different types . they also introduce an "instruct, execute, and reformat" strategy specifically designed for efficient data annotation .
Outcome:	The proposed framework surpasses open-source language models and GPT-3.5/4 on multiple evaluation metrics.

Copied to clipboard

Shang Qin, Jingheng Ye, Yinghui Li, Hai-Tao Zheng, Qi Li, Jinxiao Shan, Zhixing Li, Hong-Gee Kim

Syntactic Text Processing

Challenge:	Existing CGEC benchmarks for multi-disciplinary writing are limited . continual learning (CL) is a promising solution to handle domain-specific linguistic variation and prevent catastrophic forgetting.
Approach:	They propose a Chinese Literature Continual Learning benchmark to evaluate adaptive CGEC across disciplines.
Outcome:	The proposed benchmark includes 10,000 human-annotated sentences spanning 10 disciplines, each exhibiting distinct linguistic styles and error patterns.

Thought-Action Graph Reasoning: Faithful and Efficient Reasoning of Large Language Models via Reusing Past Experience (2026.findings-acl)

Copied to clipboard

Zhixiao Qi, Feng Huang, Yunqi Zhang, Shijie Zhang, Qingqing Sun, Yongfeng Huang, Minghu Jiang, Shuai Chen, Tianyi Zhang

SCALE: Upscaled Continual Learning of Large Language Models (2026.findings-acl)

Challenge:	Existing methods for integrating knowledge graphs with LLMs suffer from poor generalization or low reasoning efficiency.
Approach:	They propose a thought-action Graph (TAG) that decomposes LLM-KG interaction trajectories into fine-grained semantic operators and guides LLM to execute on them.
Outcome:	The proposed paradigm outperforms state-of-the-art methods on KGQA benchmarks while reducing the number of LLM calls and generated tokens.

Copied to clipboard

Jin-woo Lee, Junhwa Choi, Bongkyu Hwang, Jinho Choo, Bogun Kim, Jeongseon Yi, Joonseok Lee, DongYoung Jung, Jaeseon Park, Kyoungwon Park, Suk-hoon Jung

rSIM: Incentivizing Reasoning Capabilities of LLMs via Reinforced Strategy Injection (2026.acl-long)

Challenge:	Recent discussions suggest that further progress will come from scaling the right structure, not merely parameters or data, while preserving acquired knowledge.
Approach:	They propose a width upscaling architecture that inserts lightweight expansions into linear modules while freezing all pre-trained parameters.
Outcome:	The proposed architecture reduces severe forgetting while learning new knowledge on a controlled synthetic biography benchmark.

Copied to clipboard

Sijia Chen, Baochun Li, Di Niu